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Abstract 

THIS PAPER IS ELIGIBLE FOR THE STUDENT PAPER AWARD 
fT^ ' Scalar Quantize & Forward (QF) schemes are studied for the Two-Way Relay Channel. Different QF approaches are compared 

7-H ' in terms of rates as well as relay and decoder complexity. A coding scheme not requiring Slepian-Wolf coding at the relay is 

, proposed and properties of the corresponding sum-rate optimization problem are presented. A numerical scheme similar to the 

^SJ ■ Blahut-Arimoto algorithm is derived that guides optimized quantizer design. The results are supported by simulations. 

a ' 

^ ', I. Introduction 

[ — ] Consider a communication system where two nodes Ti and T2 wish to communicate to each other with the help of a relay 
fSJ ■ r and there is no direct link between Ti and T2. This scenario is known as a separated two-way relay channel (TWRC) IB, 

I ||2l and it incorporates challenging problems such as multiple access, broadcast, and coding with side information. 

^ I ' In this work, we focus on Quantize & Forward (QF) relaying: The relay maps its received (noisy) signal to a quantization 
l_H index by using a quantizer function Q(.). The index is then digitally transmitted to the destination nodes through the downlink 
C/3 channels. We use information theoretic arguments to find quantizers that almost maximize the sum-rate, an approach that has 
been proposed in f3] and |4| in a similar context. 

In general, vector quantizers give the best performance, but under certain conditions scalar quantizers almost maximize the 
I sum-rate. Scalar quantizers are attractive because of their design and implementation simplicity. 
K>" Our main focus is the symmetric TWRC, where both users' channel qualities are the same, both in the uplink and downlink. 
We describe the system model in Sec. |II] and in Sec. |III] we compare different achievable rate regions for the TWRC. We 
introduce a new rate region that is smaller than previous regions, but almost as large for the symmetric TWRC. The achievability 
\^ scheme does not require the relay to employ Slepian-Wolf Coding. In Sec. |IV| we look at the sum-rate optimization and 
• quantizer design problems and propose a numerical solution. Examples of optimal quantizers are given and optimal time 
sharing parameters are calculated. In Sec. |V]we evaluate the performance of the optimized system by simulations. Sec. |Vl] 
^f) concludes our work and gives future directions. 

^ ' II. System Model 

k>( ' The system has two source nodes Ti and T2 that exchange their independent messages Wi G {1, 2, ... , 2"^^}, W2 6 
j_j {1,2,..., 2"^^} in n channel uses through a relay node r. The source nodes cannot hear each other, so communication is 
Ci possible only through the relay. The communication is split into two phases: In the multiple access (MAC) phase with umac 
channel uses, both source nodes encode their messages Wi and W2 to the MAC channel inputs X""**^ and X^"**^, respectively, 
with Xi i 6 Xi, X2,t G '^2- Define a — UMAc/n as the time fraction of this first phase. The relay receives 

Yr,t ^ Xi^t + X2^t + Zr^t, ^ {1, 2, . . . , riMAc}, (1) 

where Z^^t - A/'(0,iV^) and EjX^ J < P^, E{X| J < P2. 

The relay maps 17*"*'= to a quantized representation yj,""*'^ with symbol alphabet y.^. The quantizer index is g = Q{Y""'^'^). 
During the Broadcast (BC) phase with riBC ^ n — umac channel uses, the relay transmits the codeword ^"'"^(q). The received 
signals at Ti and T2 are: 

Yj^t = Xr,t + Zj^t, t = {umac + !,...,«}, (2) 

for j G {1,2}, E{X^j.} < Pr and Zj^t ^ Af{0,Nj). Nodes Ti and T2 decode W2 and Wi, respectively, by using their own 
message as side information. Fig. [T] depicts the system setup. In the following, we often omit the time index t if we refer to 
a single channel use. In the next section we review different coding schemes and compare their performance. 
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Fig. 1: System block diagram. 



III. Achievable Rates 

A. Noisy Network Coding / Joint Decoding 

In Is), an achievable rate region was derived that matches the rates achieved with Noisy Network Coding (NNC) ||6l. The 
closure of the achievable rate region is given by the set 7<^nnc C of rate tuples {Ri, R2) satisfying 

Ri<aI{XrX\X2,U) 

Ri < {I - a)I{Xr;Y2) - aI{Yr-X\Xi,X2,U) 
R2 < aI{X2;Yr\Xi,U) 

R2 < {I - a)I{Xr;Yi) ^ aIiYr-X\Xi,X2,U) (3) 

for some p{u)p{xi\u)p{x2\u)p{yr\xi, X2)p{yr\yr) and p{xr)p{yi,y2\xr) and q > 0. It suffices to consider \yr\ < \yr\ + 2, 
\U\ < 3. 

1) Receivers: To achieve a rate tuple in 7?,nnc, the decoder must jointly decode the BC code and the MAC code in a single 
stage decoder using its own message as side information. The quantization index q is not required to be decoded. The decoder 
structure is shown in Fig. |2a] 

2) Gaussian Case: For the Gaussian case, Xi - 7V(0, Pi), X2 ~ 7V(0, P2), Xr ^ N{Q, Pr) and Yr - 7V(0, Pi+P2+Nr). 
We choose a (not necessarily optimal) quantizer yielding Yr ^ Yr + Z, where the quantization noise Z ~ J\f{0,N) is 
independent of Yr. Define C{x) i log(l + x). With U ^ ^, the achievable rate region becomes 

B. Compress & Forward 

In the spirit of classic Compress & Forward (CF), the authors of ["TI, fSl derive an achievable rate region generalizing ||2] 
using ideas from ||9J. The achievable rate region is the set TZcp C of rate tuples (i?i, R2) satisfying 

Ri <aI{Xi;Yr\X2,U), R2 < aI{X2;Yr\Xi,U) 
for aI{Yr]Yr\X2,U) <{l-a)I{Xr]Y2) 

aI{Yr-X\Xi,U)<{l-a)I{Xr;Yi) (5) 

for some p{u)p{xi\u)p{x2\u)p{yr\xi,X2)p{yr\yr) and p{xr)p{yiTy2\xr), a > 0. It suffices to consider \yr\ < |3^r| +3, 
\U\ < 4. 

1) Receivers: The coding scheme of |7| requires reliable decoding of the quantization index q at the receiver For that, the 
BC code is decoded using the own message as a priori knowledge. Knowing q, the desired message is decoded, again using 
the own message as side information. The structure of this decoder is shown in Fig. |2b] 
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2) Gaussian Case: With the same assumptions as before, i.e. Yr = Yr + Z, one obtains: 



Ri < aC 
for aC 
aC 



Nr+N 
Pl+Nr 

N 

P2+Nr 

N 



i?2 < aC 



Nr+N 



N2 

<(1-«)C(^ 



(6) 



C. Neglecting Side Information in the Downlink 

The coding scheme for CF requires Slepian-Wolf (SW) coding fTOl at the relay to reduce the downlink rate. In the symmetric 
case we do not expect this reduction to be substantial. We are thus interested in schemes without SW coding in the BC phase. 
Using random coding arguments, the achievable rates are given by the set T^noSw C K+ of rate tuples (i?i,i?2) satisfying 



Ri <aI{Xi-Yr\X2,U), R2<aI{X2-X\Xi,U) 
fox aI{Yr\Yr\U) < min{/(X^; Fa), /(X^; Fi)} 



(7) 



for some p{u)p{xi\u)p{x2\u)p{yr\xi,X2)p{yr\Vr) and p{xr)p{yi,y2\xr), a > 0. Similarly, we have \U\ < 4 and |3^,.| < 
\yr \ +3. A proof of this claim can be found in Appendix lAl 

1) Receivers: The structure of the decoder is shown in Fig. |2c] Similar to the scheme in Sect. IIII-BI two decoding stages 
are required: First the the BC code is decoded, revealing q reliably. Then q is used to obtain the desired message from the 
MAC code. In contrast to before, the own message is used only in the MAC decoder. 

2) Gaussian Case: The rate region is described by 

Pi \ r. ^ P2 



Ri < aC 

'Nr+N 

P1+P2 + N, 



R2 < aC 



Nr+N 



N 



<(l-„)n,i„|c(^),c(^ 



D. Sum-Rate Comparison for Gaussian Case 

Note that in general T^noSw C TZc¥ C 7?,nnc- We focus on the maximal sum rate Ri + R2 for each scheme in the symmetric 
Gaussian case Pi ^ P2 = P, Ni = N2 = N. This requires to jointly optimize over the quantization noise variance N and 
time allocation a. Formally, define 



i?NNc(«) 



max(i?i + i?2), (i?i, i?2) e 7^NNC 

N 



and similarly for i?^™(a) and i?j 



sum 
NoSW 



(a). The optimal sum rate is 

i?™"'(a) = 2aC 



P 



Nr + N* (a) 



(8) 



for all three schemes. The only difference is the optimal value of N for a given a, denoted by N*{a). For 7?,noSw, we have 

2P + Nr 



NoSW 



(a) = 



4 




because the sum rate is decreasing in N and ^noSw('^) '^he smallest variance satisfying the constraints. Similarly, for 72.cf, 
we have 

iVc*p(a) = ^ + . 

(l + P,/7V)'^-"'/"-l 

As A^cf(<^) < -^NoSw(o^)' we have > ^noSw(")- P^J" '^nnc, the rate expressions for NNC in Eq. (0) are either 

increasing or decreasing in N. The maximum with respect to N is thus found at the crossing point of both expressions. It is 
not hard to show that AnncC*^) ~ -^cfI"^) (^^^ Appendix iBl for a derivation). This means, given the assumption that {Yr,Yr) 
are jointly Gaussian, NNC does not provide higher sum rates than CF in the symmetric case. 

Fig. |3] shows achievable sum rates over SNR for the symmetric Gaussian case. For each curve, the value of a was chosen 
to maximize the sum rate. As expected for this setup, the difference between i?cF"(a*) = -RnncI"^*) ^'^'^ -^NoSw(q^*) small. 
The scheme corresponding to ^noSwI"^*) requires less complex relay operations. Therefore we focus on this scheme, accepting 
a slightly smaller achievable sum rate. Also note that we assume that U ~ 

IV. Quantizer Design 

A. Sum-Rate optimization for TZ^oSW 

To guide the quantizer design for DMCs and fixed discrete input distributions, we want to find the optimal 
conditional probability mass function (pmf) p{yr\yr) and time sharing coefficient a to optimize the sum-rate. With 
C :— min{/(Xr; I2), I{Xr] Yi)} as the downlink capacity, this problem can be stated as 



^N™w- sup a(l{X^-X\X2) + I{X^-X\Xi) 



a,p(yr\yr) 

subject to: %) < (1 - a)C. (9) 

Abbreviate p{yr\yr) by p and let pij p{yri\yrj)' Denote the objective as J{p) :— I{Xi; 1^1X2) + I{X2\ Yr\Xi) and define 
the function 

/(C) sup J{p). (10) 



Problem (|9]l can be stated as 



^Nrsw = supa/( ^-^C) . (11) 



Some properties of /(C) are as follows: 

1) The functions I{Xi\Yr\X2) + I{X2;Yr\Xi) and I{Yr;Yr) are convex in p{yr\yr) DU Theorem 2.7.4]. Program ((TO) 
is thus a convex maximization, a difficult problem in general. From the maximum principle illy Cor. 32.3.2], it follows 
that for the optimal p{yr\yr), I{Yr] %) = C, for < C < H{Yr). 
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2) /(C) is an increasing and concave function in C, for < C < H{Yr). The proof is along the Unes of llT3l . We refer to 
Appendix ICl for a more detailed derivation. 

Fig. m shows one example of /(C). L — \yr\ represents the number of different quantization levels. One can see that it 
suffices to consider only a relatively small L. For example, using a mapping with L = 8 quantization levels instead of i = 16 
causes a rate reduction of less then 0.03 in /(C). 

B. Computing the function I{C) 

To solve the problem for /(C) in Eq. ( fTOl l. similar to ifTTl Section 10] we write the Lagrangian: 

£(p, A, i/i, . . . , vl) = J{p)- \I{Yr;Yr) + ^ >^j^Pij, 

j i 

A > 0, where the last constraints force p{yr\yr) to be a valid pmf. The KKT conditions require 



dC dJ 



dpij dpij p{yr 

With the substitution 



\piyrj)\og -p- + V, =0, V J. (12) 



log^j :^ ~ ApS^j-) ^ = -Ap(y^j)log/ij- 



dJ\. / 9J 



it follows that £^ exp (^^^^j • But since Y.kVk2 = 1 for all j, we obtain = Y^kPiVrk) exp j , V j. 

The optimality conditions are thus given by 

P(yrOexp(^^^j 

Ptj ^ PiVrilVrj) ^ 7 o7 ^, Vl,j, (13) 



EfeP(^rfc)exp 



Xp(y,- 

PiVri) =^P{yri\yrj)p{yr3), V «. (14) 



One can solve for a conditional pmf p{yr\yr) satisfying Eqs. ( fT3l fT4l i with a fixed-point-iteration with an initial value for ^, 
similar to the Blahut-Arimoto algorithm (e.g. [11, Section 10.8]). Different initial values forp can result in different outcomes. 
In practice, we start the iteration with different initial values and store the best result. 
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C. Scalar vs. Vector quantization 

In general, a vector quantizer must be used at the relay to achieve the rate regions in Sec. |lll] An ideal vector quantizer 
results in a pmf p{yr\yr) that optimizes the problem in Eq. (flOl i. A scalar quantizer permits only deterministic single-letter 
relationships, i.e. we have 

P{yri\yrj) = 1 for some i, V j. (15) 

If ( fTSl ) is fulfilled for the optimal pmf, the quantizer function Q(.) can be directly inferred. In the saturation region in the 
curves in Fig. |4]one obtains scalar quantizers since C > log(L) > liYr] Yr). In this case the constraints for problem dTOt are 
only those for a valid pmf. As a convex maximization is optimized at one of its corner points fV2}. Cor. 32.3.2], (IT5i will be 
fulfilled. We see that the loss by using a scalar quantizer is small for sufficiently large L and proceed with this more practical 
method. 



D. Optimized Time Allocation 

I{C) in Eq. ( fTOl i captures the optimization of the pmf p{yr\yr) in problem (|9]l. To optimize also the time allocation parameter 
a, we must solve the problem in Eq. ( fTTT i. 

Proposition 1: The function al (^^^C*) is concave in a, where C is a positive constant independent of a. 

Proof 1: From property E) in Sec. HV^ we known that /'(C) := = A > and /"(C) := < 0. Define the 

function h{a) := ^^C for < a < 1. Note that 

h[a) :=^ = -i-C<Oand 
n d^h{a) 2 ^ ^ 



Further note that h {a) = —%h {a) and 

92 {al{h{a))) 



=2/ (/i(a)) ■ /i (a) + a/ {h{a)) ■ h {a)+ 



al" {h{a)) ■ h {af . (16) 

As h (a) = —^h (a), the first two summands add to zero and only the last summand remains. This term is always at most 
0, proving that ^'("^W"))) < for < a < 1. 

Prop. [T] shows that there is a unique maximizer a for the optimal sum-rate that can be found efficiently once /(C) or an 
approximation of it is known. 



V. Performance Evaluation 

Using the method of the previous section, we obtain a mapping for the following parameters: P = OdB, Pr = 9.3dB, 
Nr = N — OdB, L = 8, a = 1/3. The resulting quantizer fulfills the criteria in (ITTt (and is thus a scalar quantizer) and gives 
the sum-rate 0.29. This mapping is shown in Fig. |5] together with p{yr). 

We evaluate the performance of this mapping by means of numerical simulations. As channel codes, we use the IRA-LDPC 
codes of the DVB-S2 standard (14]. In the uplink, blocks of fci = fc2 = 7200 information bits are encoded with the rate 
0.444 (MAC Code) to blocks of umac = 16200 BPSK symbols that are transmitted from Ti and T2 to r during the MAC 
phase. The received 16200 samples at r are mapped according to the quantizer to 16200 symbols, that are represented by 
16200 • log(L) = 48600 bitfl These bits are encoded by a channel code (BC Code) of rate 3/4 to 64800 downlink code bits 
that are broadcast to Ti and T2 during the BC phase with tibc — 32400 4-PAM symbols. As a result, for the transmission 
of one block timac — 16200 symbols are used in the uphnk and ubc — 32400 symbols are used in the downlink, which 
corresponds to a = 1/3. The sum-rate of the system ((fci + fc2)/(nMAC + riBc) = 0.29 bits/channel use), the time sharing 
parameter and the transmit powers of the nodes match the optimization parameters of the quantizer At the receivers, we use 
the approach given in Sec. IIII-CI for decoding: first, the relay quantization index q is decoded without using side information. 
By using q and transmitted own symbol, the LLR of the other users symbol is calculated which is fed to the MAC decoder 

During the simulation, the noise level is varied and the packet error rates are evaluated accordingly. Fig. ^depicts the PER 
vs. uplink SNR. Recall that the noise powers used in the optimization for this example are chosen as OdB, which corresponds 
to an uplink SNR of OdB. Therefore the PER is expected to approach zero at dB. The gap to this theoretical limit is about 
0.75dB. As the DVB-S2 LDPC codes are about 0.7-1.2dB away from the Shannon limit in point-to-point channels [14| the 
simulations verify our quantizer design. Fig. |6]also shows the PER performance of the BC code (curve with circle marker) and 

'in general, Yr is not uniformly distributed and hence source coding should be performed before transmitting the indices. However, in this specific example 
H{Yr) ~ 3 and therefore source coding can be omitted. 
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Fig. 5: Numerically optimized quantizer function (dotted line) and (solid line). Note that the sohd curve represents the 
sum of the dashed curves which correspond to the pdfs conditioned on Xi + X2. 
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Fig. 6: Packet Error Rate (PER) simulations for the system with the quantizer given in Fig. |5] 

the PER of the system with perfect downlink channels (dashed line). These curves give insight: The gap between the dashed 
line and the solid line can be seen as the loss due to the imperfect BC code and the gap between the theoretical limit and the 
dashed line can be interpreted as the loss due to the imperfect MAC code and the quantizer loss. Moreover, it is interesting 
that the PER of the complete system is less than the PER of BC code. This is because even if some of the quantization indices 
are transmitted erroneously during the BC phase, they are corrected by the MAC code. 

VI. Conclusion 

We compared different QF schemes for the separated TWRC. We showed that the loss caused by neglecting Slepian-Wolf 
coding in the broadcast code can be small for symmetric setups, but it allows less complex operations at the relay and decoder 
A numerical method to maximize the sum-rate and guide quantizer design was derived and applied to special parameters. 
We observed that the loss due to scalar instead of vector quantization is small. Simulations support our results and show that 
practical schemes are close to the predicted limits. The study asymmetric scenarios is left as future work. 
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Appendix 

A. Achievability Proof 

Before we proove the rate region in (|2), we use the following usual definition of typical sequences. Notation and definitions 
essentially follow ITSl . 116 1. The analysis follows the style of |5|, |7|. 

Let x" be a sequence with each element Xi drawn from a finite alphabet X. The number of elements in x" taking the letter 
a e A" is denoted by Af(a|a;"). 

Define the typical set T"{X) as all sequences x" satisfying 

-N(a\x") ~ Px(a) <e-Px(a) e X (17) 

n 

A sequence x" S T"{X) is called e-letter-typical or just typical with respect to Px{-)- Similarly for joint distributions and 
joint typicality. 

1} Random Codebook Generation: Define timac = Cin > 0, hbc = (1 ^ o:)n > 0. 
. Choose u""*": according to Ofc^T Pui^i)- 

• Randomly and independently generate 2"^^ codewords according to Hfc^T Pxi\u{^ik\uk)- Label them a;""*'^(wi) with 
wi e {i,2,...,2"-"i}. 

• Randomly and independently generate 2"^^ codewords according to rifc™ Px2\u{^2k\uk)- Label them X2"^'^{w2) with 
W2 e {i,2,...,2"-"2}. 

• Randomly and independently generate 2""*'=^'3 codewords according to Hfe™ Py^iuiyrkluk)- Label them with 
i e {l,2,...,2""*c-«Q}. 

• Randomly and independently generate 2™"*'^^'' codewords according to Y[k=i P^ri^rk)- Label them with i £ 
{1,2, ...,2""*^-««}. 

Reveal all these codebooks to all nodes. 
2) Coding: 

1) In order to transmit message wi, node 1 sends 

2) In order to transmit message W2, node 2 sends x^'^^'^ {'W2) ■ 

3) When the relay receives y;?"*^ it looks for the first index i such that (y""*S G TJl"''''iYr,Yr\u""^''). If it does 
not find such an index, i — 1. The relay transmits 

4) When node 1 receives y""*^, it looks for the unique i such that y"""^) £ 7^"'"^(Xr, Yi). If no or more than one 
such i is found, node 1 chooses W2 = 1. Node 1 thus knows y""*'^(i). 

5) Node 1 decides for the unique W2 that satisfies {x"""" (wi) , x^""" {W2) , yr"^"" (i)) € XT"'' {Xi, X2,Yr\u''"''''). If no or 
more than one such index is found, node 1 sets W2 = 1. 

6) Steps 4 and 5 similarly for node 2. 

3) Error events: 

£1 is the event that coding step 3 fails. 

That is, $i e {1,2, . . . ,2"-^<5} such that (y""*S e 7^""*^(i;, i^r|u""*^)- This is not an intrinsic error event but 

simplifies the analysis. 
£2 is the event that coding step 4 fails. 

That is, J/D ^ ^1) or yD ^ XTiXr, Yi) with ^ ^ j. 

£3 is the event that coding step 5 fails. 

Assume a;""*'^(u'i) and X2"'^''{w2) are sent and the relay chooses the index i such that (y"™, £ 

7^nMAc(y;, y^|u"MAc), denotes the event that {xT'"''{wi),xT"'{w2),yT''''{^)) ^ TJ^'-'^iXi, X2,Yr\u""'''') or 

{xT'"'' M, a;2"*" iw2 ) , y?"*" («) ) G Te""*' (^1 : ^2 , I U""*" ) with ^2 7^ W2 • 

The error events and achievable rates for node 1 are similar 

It is not hard to check that the probability of error P{£) is bounded by the probability of the union of these error events, 
i.e. ^ ^ 

Pi£)<pl[j£k] <i2^>^- 

\k=l / k=l 

4) Error analysis: Event £1: 

Joint typicality of requires y;'"*^ ^ 7;^, (rr|u""*c). However, P{yl!-""'' ^ Te^,{Yr\u'^»"^)} for 

"MAC 00 . 
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As the codewords y'^"^'^[i), i £ {1, 2, ... , 2""ac-Rq} ^re drawn i.i.d.. 



E 



2'^MAC-f^Q 



(a) 

< 



E 



Y^"MAC|(jnMAC IS/r I" 



I _ 2-«MAc(/(i'r;^'.l"""*C)+5(ei)) 



2"MAC-'*Q 



< 



^ynMACj^nMAC^ gjj.p ^_2"MAc(-RQ-(/(i'r;J'r|;7)+(5(ei)))~j 



< exp 



(18) 



(19) 



(20) 



(21) 



where 

(a) follows from the joint typicality lemma |T6l bounding the probability that a given y""*'^ is jointly typical with a randomly 
independently sampled codeword by 



pl^yKMAC ^"MAC^ g -y-nMAC^y Y | u""*"^ ) } > 2^""*'^ ' '^•'"'"'''■'^1 

(fe) follows from (1 — x)" < exp(— n • x) ifTTl . 
(c) is due to Ey:?«ACg^^^^(^^|„„„^c) -Py;'MAc,[,„„Ac(2/""*1w""''") < 1 
In the Hmit it follows that P{£i) for n ^ oo for ei > ci' > if 



(22) 



ai?Q > aIiYr;Yr\U). 



(23) 



Event f 2 



This is the classical proof of the channel coding theorem. We can split this event in two subevents £21 and £^22 with 
P{£2) < P{£2i) + P[£22)- Define 

S21 ■.H(xT-{i),yT') ^VT'{Xr,Y,)} 
£22 G T,T{Xr,Y,)} for some j ^ i 

Since (X;Bc(i), y»Bc) _ -Qnec^ Px.,n|c/(a;.fc, yulufc), by the law of lai-ge numbers 

P(^2i) for riBc — > oo- 



(24) 



For £22, by symmetry, we focus on the case that was transmitted. 

Note that (X;B^(j),yr') rife" PxAu{xrk\uk)PY,\u{yik\uk) for i ^ j. 

2"mac-Rq 

P(f22)< E n«"(j),yr)e7;:-(x.,ri)} 

<^ 2""A':-'^Q2""'''^(-'^(^'-'^i^"''(''^^' = 2""(i""(-'"(^'-''*'i)^''(^2))-aflQ) 

where 

(a) follows from the joint typicality lemma stating 

^'{«'''(j),2/r') e 7;!^'''^(Xr,yi)} < 2-"»<^(^(^-^i)-'^(^^». 

Concludingly, in the limit P{£2) for n — > 00 for £2 > if 



(25) 



uRq < 1 -a/(X^;Yi). 



(26) 



Event £3: 

Again, we can split this into two subevents and £32 with P{£z) < -P(^3i) + P{£32)- Define 
£31 :={(xr-(u.i),x^'"-(u.2),yr'^(i)) ^ 7;'3'"-(Xi,X2,i;|u""-)} 
£32 :={(x;'"*^(w;i),x^"*^(u;2),yr"''(«)) e 773'"*^ (Xi, X2, for some W2 ^ W2 

From the coding scheme, i;""*^ = 2(1;""*'^) and thus {Xi,X2) Y,. Y,.. 
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For £31, by the law of large numbers, P{(a;?;'"*^ (wi), a;^"*^ (^2), y""*'^) e 7^","*'^ (Xi, X2, Fr-|u""*'^)} ~> 1 for riMAC ^ 00. 
By the Markov Lemma 1161, typicahty of (wi), (^2), 2/""*'') impUes 



P{(xr*^(zi;i),xr-(«;2),2/?"*^yr*^(»)) e T;^- (Xi , X2 , , | u""- ) } ^ 1 for t^mac ^ 00 
for < £3' < £3* . A direct consequence is that 

F{(x5'"*^(u;i),a;^"*^(w2),yr*'(*)) ^ T;^*^ (Xi , X2 , | u""*^ ) } ^ for timac ^ 00. 



(27) 



(28) 



for < £3 < £3.. 
For £32: 

This part is similar to the multi-access channel. 
Note that (X{'"*^(«;i),X2""^^(zz;2),^;""*^(j)) ~ 111=1 Px,X\ui^^k,yrk\uk)Px,\u{x2k\uk) for W2 W2. By introducing the 
random variable T/""ac — (X""*'^ , 1^""*'^ ) we can use the same steps as for error event £22 and conclude that 

2"-R2 

^^(^32) < ^{(4"*'(^«2),t'"«*") e 7;;j"*^(X2,F|w""*^)} 

2"-'^22-nBc(/(X2;y|C/)-5(e3)) ^ 2""(l-"(-f(-^2;V|C/)-6(e3))-fl2) (29) 

where 

(a) follows from the joint typicality lemma. With I{X2] V\U) — I{X2] Xi,Yr\U) — I{X2] Yr\Xi, U) due to the independence 
of Xi and X2 given U, in the limit ^(^3) for n — > 00 if 



R2<aI{X2]Yr\Xi,U). 



(30) 



5} Cardinality ofU: This derivation closely follows the proof in [17, Chapter 3.2]. 
The achievable rate region can be written as 



Ri<Y. P(u)aI{Xi-Yr\X2, U = u) 
ueu 

R2 < ^p(u)a/(X2;i;|Xi,{7 = M) 

ueu 

0<Y^ p{u) (1 - a)I{Xr] Y2) - aI{Yr] Yr\U = u) 
ueu 

< ^ p{u) \{l ~ a)I{Xr: Yi) - aI{Yr] Yr\U = u) 



ueu 



Yp{u)gi{p{xi,X2\u)) 
ueu 

Yp{u)g2{p{xi,X2\u)) 

ueu 

Yp{u)g3{p{xi,X2\u)) 
ueu 

Yp{u)gi{p{xi,X2\u)). 



(31) 



ueu 



The rate region can be interpreted as a subset of the convex hull of the region in the 4-dimensional space spanned by 
the functions {gi[p{xi,X2\u)),g2{p{xi,X2\u)),g3{p{xi,X2\u)),gi{p{xx,X2\u)), which only depend on the conditional pmf 

p{xi,X2\u). 

Let S denote the set of all such points for each choice of the compact set p{xi)p{x2)p{yr\xi,X2)p{yr\yr)p{xr)p{yiTy2\xr)- 
Precisely, 

S= IJ {gi{p{xi,X2)),g2{j){xi,X2)),gi{p{xi,X2)),gA{p{xi,X2))} 

As continuous image of a compact set, S is connected. Define C = conv(5): By the Fenchel-Eggleston strenghtening of 
Caratheodory's theorem (e.g. [16, Appendix A, C]), every point in C can be obtained by taking a convex combination of at 
most 4 points in S. As the rate region is a subset of C, it follows that \U\ < 4. 
6) Cardinality ofyr-' Similarly, the rate region can be written as 
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i?i < ^ p{yr)a \h{Xi\X2,U) - H{Xi\X2,UX 



X! Piyr)9l{p{yr\yr)) 



R2< Piyr)a\H{X2\Xi,U)-H{X2\Xi,U,Yr^yr) 



X] Piyr)92{p{yr\yr)) 



0< ^ p{yr)\{l'a)I{Xr;Y2)^a(H{Yr\U)~H{Yr\uX^Vr) 



X! Piyr)93iP{yr\yr)) 



(32) 



(33) 



Additionally, the following \yr \ — 1 conditions have to be met: 

p{Yr = yri) = ^ P{yr)p{Yr yri|i^r = 2}r) := X! ^'(y'')5i+4(p(2/r l^r)) Vi = 1, . . . , |X | " 1 

v^&y^ Vreyr 
Again, the rate region can be interpreted as a subset of the convex hull of the \yr \ + 3-dimensional space spanned by the 
functions gi{p{yr\yr)), ■ ■ ■ : 9\yr\+3{piyr\yr)), only depending on the conditional pmf p{yr\yr)- Precisely, let S denote the set 
of all points spanned by gi{p{yr\yr)), ■ ■ ■ 1 9\yr\+3ip{yr\yr)) for each choice of the compact set p{yr\yr), i.e. 

U i9iipiyr\yr)),---,9\y,.\+3{p{yr\y7-))} 

PivAVr) 

As continuous image of a compact set, S is connected. Define C ~ conv(iS): By the Fenchel-Eggleston strenghtening of 
Caratheodory's theorem (e.g. [16, Appendix A, C]), every point in C can be obtained by taking a convex combination of at 
most \yr \ + 3 points in S. As the rate region is a subset of C, it follows that |3^r| < |3^r| + 3. 

B. Derivation of the claims in Sec. \III-D\ 

For 7?^cF and 7?.noSw the expression in Eq. ([8]l is straight forward. For 7?^nnc, the sum rate problem for the symmetric case 
can be written as 



i?^7c(a) maxmin{2aC 

N 



P 



Nr+N 



P, 



.2 (l-a)CP^ } 



N 



Nr 



N 



The first expression inside the minimum is strictly decreasing an convex in N. The second one is strictly increasing and 
concave in N. Implicitly, we always assume that the second expression is > 0. The function represented by the minimum 
is thus quasi-concave in N. In particular, the maximal value is attained where both expressions inside the minimum are the 
same. For that. 



2aC 



P 



a log 



'Nr+N + P N + Nr 



Pr 



Nr+N 



N J 

Nr+P 

N 

^NNC(") 



N 



(l-a)log(l + ^ 



Nr 



N 



Pr 

'+N 



(1-q)/c 
P + Nr 



{1+ Pr/N)(^-°')/" - 1 



(34) 



which equals /V2p(a) 



C. Derivation of the properties of Sec. I/V-AI 

We restate the properties with full explanation here 
• I{C) is upper bounded by 

I{C) < iI{XuYr\X2) + IiX2;Yr.\Xi)) . 



(35) 



with equahty if C > H{Yr). This is due to I{Yr; Yr) < H{Yr), with equaUty if H{Yr\Yr) = 0. Now, the upper bound in 
Eq. ^ is met if I{Xi:Yr\Yr, X2) = 0, i.e. 



I{Xi;Yr\Yr,X2)^H{Yr\Yr,X2)-H{Yr\Yr,Xi,X2) 



(36) 
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• The functions I{Xi; Yr\X2) + I{X2;Yr\Xi) and I{Yr; Yr) are convex in p{yr\yr) IfTTl Theorem 2.1 A}. Computing /(C) 
thus requires maximizing a convex function over a convex set, a difficuh problem in general. From the maximum principle 
im Cor 32.3.2], it follows that for the optimal p{yr\Vr), I{Yr;Yr) = C. 

• /(C) is an increasing and concave function in C, for < C < H{Yr). The first part follows by the consideration before. 
Thus, for < Ci < C2 < H{Yr), I{C\) < /(C2). As a consequence, one can restrict the optimization to the constraint 
I{Yr; Yr) = C. The second part will be proved in the following. 

Recall that we abbreviate p{yr\yr) by p: We rewrite the problem, similar to llT3l and H: 



(37) 



/(C) ^SUp [H{Xi) + H{X2) - H{Xi\Yr,X2) - HiX2\Yr,Xi) 
s.t. H{Yr) - H{Yr\Yr) = C, < C < H{Yr). 

By dropping the constant terms, an equivalent problem (in the sense of the same optimal argument) is given by 

H{X2\Yr,Xi)) , 
s.t. H{Yr\Yr) = X, 0<X<H{Yr). 



F{x) ■.= mi{H{Xi\Yr,X2) 
p V 



(38) 



We investigate properties of F{x). 

In the following, we often use a vector representation of marginal probability distributions. The distribution of a general 
random variable Z, p{z) is equivalently represented by the column vector Pz G ^\z\ in ths |2|-dimensional probability simplex 
A|2:|, describing an (|Z| — 1) -dimensional space. The i-th coordinate is denoted by pz^i — p{Z = Zi). 

Therefore, let e ^IJ't-I' Pyr ^ represent the marginal distribution p{yr), p{yr), respectively. 

Let B = [bi, . . . , hi] be a \yr \ x L stochastic matrix with hi e A|^^[ in the i-th column. 

Introduce the random variables 

• 17 with marginal distribution Pyj = Bpy^ — ^f^i piyri)bi, 
. X[ with marginal distribution p(x'y) = X)l=i Pixij\yri)piy'ri)^ V j = 1, . . . , | A'l |, 
. X!^ with marginal distribution ^(xj ) = X^Ei P(.^2j\yr2)p{y'ri)' V j = 1, . . 



In general, the matrix B corresponds to p{y!r\yr)- Clearly, if B is equal to to p{yr\yr 
and p{x'2) = p{x2). 
One can write: 



then p{y'^) = pijjr), p{x[) = p{xi) 



4=1 

^^H{Y;\Yr)^ 



L 

E 

4=1 



p{yri)H{Y^\Yr = y„) 



L 

E 

4=1 



p{y„)h\yjh,)<l0g{\yr\) 



with /i|y^|(p) = 2_^Pj log(Pj) as the entropy function. 



(39) 



(40) 



= H{X[\X^,Yr) + H{X!2\X[,Yr: 



<ios(\Xi\) 

The problem in Eq. ( |38] | can be stated as 



HiX[\X!„Yr - y„)HiX!2\X[,Yr - y„) 



<log(|A:'2|) 



:=E^'(^")-9(bO <log(l'^i| • \X2\ 



F{x) 



inf 77, 



S.t. e 



(41) 



(42) 



Now define the mapping S A|j;^,| (b^, (b^), ^((bi)) Remember that A|j;^| is (|3^r| — 1) -dimensional, so the 
polytope A|3;^| x [0, log(|3^r|)] x [0; logdA:"! I • IA2I)] is (|3^r| + l)-dimensional and the mapping assigns points inside this 
polytope for each b^. Let S denote the set all all these points. As h^y^^{hi) and g{hi) are continuous functions of b^ ifTSl 
Chapter 2.3], S is compact and connected. 
Define C as the convex hull of S, i.e. C ~ conv(5). 

Lemma 1: The set of pairs (Py;, r]) defined in Eqs. ([391-I4TI) is precisely C, for all integers L > 0, Py^ G Al, hi e A|j;^|, 
i^l,...,L. 

Proof 2: |13, Lemma 2.1] by definition of the convex hull. 
Proposition 2: The function F{x) is convex in x for < a: < H{Yr). 

Proof 3: The function of interest F{x) is the minimum of rj for which Py/ = Pj,^ and £, = x. 
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Define Cp as the the projection of the intersection of C with the convex and compact set defined by Pj,/ = Pj,,, onto the 
plane (Cv) C R^- Precisely, Cp^^ = Proj{C n Cp^J, where Cp^^ = {{Py'^,£.,v) C Ml^-l+i|py;, = p^J. Note that £p^_, is 
convex and compact. As convexity is preserved under intersection |19, Sect. 2.3.1] and projection onto coordinates [19, Sect. 
2.3.2], the set Cp is also convex and compact. That is, the infimum in Eq. ( |38] l can be attained and is thus a minimum, 
provided that the intersection C D Cp^^ ^ 0. Like in 1131 , for L = 1, bi = p^^, p{yri) = 1 it follows that Pyr — Py^ and 
f = h\y^\{PyJ = H{Yr). Choosing L = Py, = Py,, B = [bi, . . . ,b|j,_,|] = I^y^^, it follows that Pj^^ = Pj^^ and f = 0. 
By the convexity of C, there must be points for which Pyi = Py^ and (, = x for < a: < H{Yr). Thus, the intersection is 
never empty. 

As Cp is convex and compact and F{x) is the boundary of this convex set, F{x) is itself convex, as illustrated in Fig. |7] 
(similar to Fig. C.2]). 



iog|A'i| + iog|;f2 



F{x) 



H(Xi\X2,Y^) + 
H(X2\Xi.Y^) 



increasing C 

Fig. 7: Visualization of Cp^^ and the function F{x) 

Corollary 1: I{C) is a concave function in C, for < C < H{Yr). 
The proof follows by Proposition |2] 

D. Interpretation of Lagrangian 

It is known from Cor [T]that /(C) is concave in C corresponding to the quantizer rate /(l^; Yr), for < C < H{Yr). Let 
{C\,I\) be the point on the /(C) curve at which the tangent has the slope A. As /(C) is nondecreasing, A > 0. The tangent at 
this point intersects with the y-axis at I\ — \C\, as illustrated in Fig. |8] Recall that we abbreviate p{yr\yr) by p: Let (Cp' , Ip' ) 
be another point with Cp' = liYj.; Yr) and Ipi — J{p') corresponding to some conditional pmf (cond. pmf) p' ^ p*, i.e. not 
lying on the /(C)-curve. The line through {Cp' , Ip') with slope A intersects the y-axis at Ip' — XCp'. Due to the concavity of 
/(C), all intersections of lines with a given slope A lie below I\ ~ XC\. To find the optimal axis intercept I\ — XC\ one can 
write: 

/^-_ACa= max {Ip - XCq} ^ max {C{p,X)}, A>0 (43) 

cond. pmf p cond. pmf p 

with C{p, A) corresponding to the Lagrangian of the problem (dropping the equality constraints for p which are captured in 
the constraint set here). 

Running the optimization routine in Sec. IIV-B] for a Lagrangian multiplier A should return the point on /(C) with slope A. 
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